Transaction session for wearable processing device

ABSTRACT

A transaction session is established directly or indirectly between a wearable processing device and a cloud-based server of a store. During the session, items are recognized by placing the items in a field-of-view of a front-facing camera of the device. Item recognition does not require item barcode identification. A depth sensor associated with the camera creates a three-dimensional mapping of a given item. The mapping and image features are processed to uniquely identify the item even when the item is associated with a same category of items. Customer input during the session can be achieved through gestures (hand, eyes, head, fingers, etc.) and/or voice commands. The customer input is translated and mapped into transaction interface commands/options and processed during the session to select items, delete items, view a transaction receipt, identify a quantity of items, obtain item details for a given item, etc.

BACKGROUND

As technology advances and consumers embrace it in all aspects of theirlives, many retailers have taken advantage of this phenomenon withtechnology offerings that make it easier for their customers to interactwith the retailers and transaction with the retailers.

For example, most retail stores now have Self-Service Checkouts (SCOs)where customers of the stores can self-checkout. Customers utilizingSCOs typically have to pick items from the store shelves, carry them tothe SCOs, scan the item barcodes at the SCOs, and pay for the goods. Theproblem with this approach is that the customers have to handle theitems multiple times before checking out (pick from shelves, place incart, remove from cart, scan at the SCOs, bag the items, etc.).

Consequently, many retailers now offer mobile applications accessiblefrom their customer phones that permit their customers to scan itembarcodes as they shop in the stores and place scanned items in bags of acart or a basket. Scan as you shop applications have streamlined thecustomer experience within the stores. However, these applications stillhave a number of problems, which have limited customer adoption of thistechnology.

The scan as you shop applications require the user to actively operatetheir mobile phones as they shop. This creates a usability issue becauseone customer hand has to hold a picked item while the other customerhand has to operate the phone and interact with the mobile applicationduring shopping. Customers struggle to carry their personal belongings,deal with small children and/or push a cart (or carry a basket) whileoperating scan as you shop applications on their phones. Many customersfind this experience too cumbersome and difficult.

Additionally, most scan as you shop applications require the customersto properly orient a held item so that its barcode is placed in thefield of view of the phone's camera for properly identifying andrecording an item identifier for the item.

As a result, there is a need for improved scan as you shop applications,workflows, and interfaces.

SUMMARY

In various embodiments, a system and methods for transaction sessionswith wearable processing devices are presented.

According to an embodiment, a method for managing a transaction sessionwith a wearable processing device is provided. A connection to acloud-based store server is requested during a shopping trip of acustomer to a store. A wireless transaction session is established withthe cloud-based store server based on the requested connection. Itemimages placed within a field of view of a front-facing camera of awearable processing device worn by the customer are captured during thetransaction session. Item identifiers and item information are obtainedfor items associated with the item images based at least on the itemimages. Gestures of the customer are translated during the transactionsession into customer selections, customer options, andcustomer-initiated commands associated with a virtual shopping cartmaintained by the cloud-based store server during the transactionsession; the virtual shopping cart comprises the item identifiers anditem information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a system for managing transaction sessions withwearable processing devices, according to an example embodiment.

FIG. 2 is a diagram of a method for managing a transaction session witha wearable processing device, according to an example embodiment.

FIG. 3 is a diagram of another method for managing a transaction sessionwith a wearable processing device, according to an example embodiment.

DETAILED DESCRIPTION

FIG. 1 is a diagram of a system/platform 100 for managing transactionsessions with wearable processing devices, according to an exampleembodiment. It is to be noted that the components are shownschematically in greatly simplified form, with only those componentsrelevant to understanding of the embodiments being illustrated.

Furthermore, the various components (that are identified insystem/platform 100) are illustrated and the arrangement of thecomponents are presented for purposes of illustration only. It is to benoted that other arrangements with more or less components are possiblewithout departing from the teachings of conducting, operating, andmanaging transaction sessions via a wearable processing device,presented herein and below.

System/platform 100 (herein after just “system 100”) provides aprocessing environment by which a customer engages in a transactionsession with a retail store's server via an improved and seamlessinterface associated with a wearable processing device (such asglasses/goggles/headsets) during a shopping trip at the store. Thewearable processing device permits the customer to view theirsurroundings unobstructed through the lenses of the wearable processingdevice. A camera of the device captures what the customer is looking atwithin the physical surroundings during the session and permits seamlessand handsfree identification of items being handled by the customer.Within a portion of one or both lenses, superimposed item andtransaction information is rendered for viewing, selection, acceptance,and/or modification by the customer using Augmented Reality (AR)techniques.

The wearable processing device may have its own independent wirelessnetwork connection with the store's server or may utilize a wirelessconnection to the store's server via a second device in possession of orin proximity to the customer. Item recognition of items captured withinthe field-of-view of the lenses do not have to include images of theitems' barcodes. Moreover, machine-learning models may be continuouslytrained to identify hard to recognize items, such that the itemrecognition is continuously improving on item recognition accuracy.

The customer may complete the session and pay for items of thecustomer's virtually maintained cart (with the store's server) withoutthe customer having to operate any additional device during the shoppingtrip or during the transaction session, such as the customer's phone, aSelf-Service Terminal (SST) during self-checkouts, and/or aPoint-Of-Sale (POS) terminal during cashier-assisted checkouts.

A variety of embodiments and operational features of system 100 are nowdiscussed with reference to FIG. 1 .

As used herein, the terms “user,” “consumer,” and/or “customer,” may beused interchangeably and synonymously herein and below. This refers toan individual who is wearing a wearable processing device and is engagedin a transaction session with a store's server during a customer'sshopping trip with a store.

The system 100 comprises a cloud/server 110, a wearable processingdevice 120, and a user-mobile devices 130 (optional configuration ofsystem 100).

Cloud/server 110 comprises at least one processor 111 and anon-transitory computer-readable storage medium 112. Medium 112comprises executable instructions for a transaction manager 113 and anitem recognizer 114. The executable instructions when provided toprocessor 111 from medium 112 cause the processor 111 to performoperations discussed herein and below with respect to 113-114.

Wearable processing device 120 comprises at least one processor 121, atleast one camera 122, an accelerometer 123, a wireless transceiver 124,zero or more sensors 125, and a non-transitory computer-readable storagemedium 126. Medium 126 comprises executable instructions for a HumanInput Device (HID) manager 127, a session agent 128, and a transactionmanager 129. The executable instructions when provided to processor 121from medium 126 cause the processor 121 to perform operations discussedherein and below with respect to 127-129.

Mobile device 130 (optional) comprises at least one processor 131, oneor more sensors 132, wireless transceiver 133, and a non-transitorycomputer-readable storage medium 134. Medium 134 comprises executableinstructions for a transaction manager 135 and a session manager 136.The executable instructions when provided to processor 131 from medium134 cause the processor 131 to perform operations discussed herein andbelow with respect to 135 and 136.

System 100 initially detects a connection request to cloud/server 110for a transaction session associated with a customer's shopping trip toa store of a given retailer. Detection can occur or can be requested ina variety of manners.

For example, a Global Positioning Satellite (GPS) sensor 125 may reporta current physical location of wearable processing device 120, whichtransaction manager 129 maps to a given store's location. The lenses (ARdisplay) of device 120 superimposes a question overlaid on thecustomer's view of their surroundings as a notification stating, “Starta Shopping Trip with Store X?” HID manager 125 monitors a front facingcamera 122, a rear facing camera 122, the accelerometer 123, and/or thesensors 125 (such as a microphone, pressure sensors on the rims ofglasses 120, and/or a button located on the rims of the glasses) forcustomer-provided input. An affirmative response causes session agent128 to activate wireless transceiver 124 and establish a wirelessconnection 140 between transaction manager 129 and transaction manager113 of the store's cloud/server 110.

In another example, glasses 120 maintains a continuous wirelessconnection 160 using wireless transceiver 124 with mobile device 130 viawireless transceiver 133 of mobile device 130. (Note mobile device 130can be in a pocket, purse, of bag of the customer and does not have tobe actively operated by the customer for this continuous wirelessconnection 160.) Mobile device 130 uses a GPS sensor 132 to detect amobile device's physical location being associated with a physicallocation of the store. Session manager 136 establishes a wirelessconnection 150 (cellular, Wi-Fi, etc.) between transaction manager 113of the cloud/server 110 to transaction manager 135 of mobile device 130.Based on an existing connection 160, session manager 136 sends thenotification to session agent 128 for presenting the question as towhether the customer wants to start a shopping trip (as discussed abovein the previous example) on the AR lenses/display of glasses 120 to thecustomer. An affirmative response from the customer as detected by HIDmanager 126 causes session agent 128 to interact with transactionmanager 135 during the transaction session being conducted betweentransaction manager 135 and transaction manager 113 over wirelessconnection 150. In this embodiment, session manager 136 acts as a proxybetween a user-facing interface of transaction manager 135 of mobiledevice 130 and HID manager 125 via session agent 128 over wirelessconnection 160.

In still another embodiment, a Quick Response (QR) code placed on adisplay at the front of the store or placed on a label of a surfaceproximate to the store's entrance is captured by the customer via afront-facing camera 122 of glasses 120 or via a camera (not illustrated)of mobile device 130. The session agent 128 or session manager 136establishes connection 140 or connection 150 between glasses 120 ordevice 130 based on reading the QR code and the notification presentedon the AR lenses/display of glasses 120 for an affirmative response bythe customer in manners similar to what were discussed above in theprevious two examples (first example, glasses 120 directly interactswith transaction manager 113 of cloud/server 110 for connection 140during the session using transaction manager 129; second example,glasses 120 indirectly interacts with transaction manager 113 ofcloud/server 110 over connection 160 via transaction manager 135 ofdevice 130 and device 130 directly interacts with transaction manager113 via transaction manager 135).

Once an affirmative response to the shopping trip notification isreceived from the customer via HID manager 127 over connection 140 orover both connections 160 and 150, a transaction session for acustomer's shopping trip with a store is established between transactionmanager 129 and transaction manager 113 (direct connected session) orbetween transaction manager 135 and transaction manager 113 (indirectconnected session).

Session agent 128 begins monitoring HID manager 127 events foruser-directed input and begins monitoring images appearing within afield of view of front facing camera 122 for item images of items afterthe transaction session is established. Interaction for a directconnected session may cause transaction manager 129 to download and toprocess cloud/server-hosted Application Programming Interfaces (APIs) totransaction manager 113. The APIs may already be available and in use bytransaction manager 135 for an indirect connected session.

During the transaction session, when a customer desires to purchase anitem, the customer picks the item off a shelf and places it within thefield of view of front facing camera 122.

In a directly connected session, session agent 128 samples features ofthe item image and sends the features over connection 140 to itemrecognizer 114. Item recognizer 114 scores the features and matches tocandidate items, when a score of a candidate item is above a predefinedthreshold, item recognizer 114 returns the candidate item's code totransaction manager 113 and transaction manager 113 provides totransaction manager 129. Transaction manager 129 displays the item nameand image within the AR lenses/display of glasses 120 to the customer.When none of the candidate items for the features of the item image areabove the predefined threshold, a top N number of item identifiers forthe candidate items are provided by recognizer 114 to manager 113.Manager 113 provides to manager 129 and the candidate item images anddescriptions are rendered within the AR lenses/display for selection bythe customer. The customer may select one of the candidate item imagesthrough input identified by HID manager 127. For example, a thumbs upplaced in front of a displayed candidate item image is interpreted as auser selection for the item code associated with the candidate itemimage. Each candidate item image may be presented sequentially to thecustomer within the AR lenses/display and browsed by the customerswiping a hand to the right or left within the field of view of frontfacing camera 122 to see a previously presented candidate item image(swipe right) or see a next candidate item image (swipe left).

Some items may not be able to be completely recognized based on thecaptured item image. For example, different flavors of a brand-namedrink, in such cases item recognizer 114 provides the differentsubcategories (flavors) for manager 113 and the user is presented thechoices via the AR lenses/display for selection via HID manager 127.

When an item is recorded for the transaction session through automaticrecognition or through user-selection via HID manager 127, the item codefor the selected item is stored in a virtual cart being maintained forthe shopping trip and the transaction session by transaction manager 113for the customer. Additionally, when an item code is added a corner ofthe AR lenses/display is updated to show a current running price totaland a total quantity of items associated with the virtual cart, suchthat the customer is always aware of a current price of the itemsrecorded in their virtual shopping cart during the transaction session.

In an indirectly connected session, the above-described workflow of itemrecognition, selection, and updating of the virtual shopping cart issimilar; however, the workflow is processed over connection 160 (glasses120 to mobile device 130) and connection 150 (mobile device 130 tocloud/server 110). Here, session agent 128 interacts with sessionmanager 136 to proxy a user-facing interface to transaction manager 135on the glasses 120 using HID manager 127 to capture and process userinputs.

In an embodiment, glasses 120 includes a depth sensor 125, which allowsthe features of an item image taken to be taken in three dimensions (3D)to capture an item's depth, height, and width. A 3D mapping produced bycamera 122 can be derived using the depth sensor 120 with the 3D mappingprovided as the item features to item recognizer 114. This allows itemrecognizer 114 to distinguish between different sized items of the sametype, such as an 8-ounce drink versus a 12-ounce drink; a smaller sizedpiece of fruit versus a larger sized piece of the same fruit. Moreover,this provides fine grain item recognition which has heretofore not beenpossible in the industry.

In an embodiment, when a candidate item's image is captured by frontfacing camera 122 and that image includes a completed item barcode, itemrecognizer 114 can quickly identify the item code, item image, and itempricing/details and return to manager 113, which is then used to updatethe virtual cart causing the update to be reflected in the cart summarypresented to the customer in a corner of the AR lenses/display ofglasses 120.

HID manager 127 can map a variety of user gestures and other userprovided inputs during the transaction session to options associatedwith a user-facing interface of transaction manager 129 and transactionmanager 135. For example, when camera 122 detects a customer handgesture to the side of one of the AR lenses, HID manager 127 maps thisgesture to a user-facing interface option associated with a request bythe customer to view the total list of items scanned and recorded(current transaction receipt) in the virtual cart for the session. Ahand gesture that moves from a center of the two lenses to the side ismapped to a user-interface request/option to minimize the transactionreceipt back to a transaction total and total quantity of items in acorner of the AR lenses/display of glasses 120. To scroll thetransaction items in the receipt, the customer moves their hands upwardor downward. The displayed items are numbered or labeled when presentedwithin the AR lenses/display, a microphone sensor 125 may capture thecustomer speech that references a specific number in the list and thecustomer and see more details or delete the item associated with thespecific number audibly (which HID manager 127 maps the audible speechinto a text command recognized by the user-facing interface of manager129 and/or manager 135). A quantity of any given item can be increasedby the customer placing a same item back in the field of view of the ARlenses/display for each number of the items or the customer can placethe item in the field of view and audibly state quantity is N.

In an embodiment, HID manager 127 tracks events from sensors 125 andaccelerometer 123 of glasses 120 and maps predefined gestures and/oraudible spoken words into commands recognized by the user-facinginterface of manager 129 and manager 135. The gestures can be a head nodup, a head node down, head swipes side to side, a head tilt up down orside to side, which are detected by accelerometer 123; eye movements ofthe customer detected in images of a front-facing camera 122 of glasses120 while eyes of the customer are being tracked during the transactionsession; spoken words detect recorded as audio by a microphone sensor125; finger taps on a surface of the rims of the glasses 120 detected bya pressure sensor 125; pressed buttons manufactured in the rims of theglasses 120 and associated with a button sensor 125, etc.

The customer can finish their session and corresponding their shoppingtrip through payment for the virtual shopping cart. Payment workflowscan occur in multiple manners. Payment processing can be initiated bythe customer capturing the field of view of front facing camera 122 a QRcode being displayed on an SST, this causes the QR code to be decodedand an SST identifier for the SST communicated to transaction manager113. Manager 113 identifies which device that communicated the SSTidentifier, glasses 120 or device 130 and identifies the virtual cart.Using the SST identifier, the SST is placed in payment mode for thetransaction details associated with the customer's virtual cart andpayment can be received in a number of manners. The total transactionreceipt is displayed on a display of the SST and displayed within the ARlenses/display of glasses 120. The customer may nod or may audibly sayyes and a registered payment method for the customer is processed forthe payment. In another case, the customer may simply say ready forpayment audibly, which HID manager 127 translated into a pay now optionassociated with the user-facing interface of manager 129 and 135. If thecustomer does not have a registered payment method in their profile orif the customer wants to use a different payment method from an existingregistered payment method, the customer may place a payment card in thefield of view of the AR lenses/display and the image of the payment cardis processed using Optical Character Recognition (OCR) and provided totransaction manager 113 as a new payment method.

In an embodiment of an indirect connected session, session manager 136keeps the user-facing interface of transaction manager 135 insynchronization with actions taken by the user during the session andthe current state of the session. This allows the customer/user toswitch back and forth between providing gestures or speech as input viaglasses 120 and touch-based input for the user-facing interface oftransaction manager 135 on device 130. The customer/user can perform orprovide commands that the user may be unfamiliar with through glasses120 using the touch-based input for the user-facing interface of manager135 on device 130. It is noted that any available command to theuser-facing interface of manager 135 is available to the user throughthe gestures or audio inputs on glasses 120, such that this option isavailable as an alternative input mechanism to the user but does nothave to be used by the user.

In an embodiment, item recognizer 114 utilizes a trainedmachine-learning model for item recognition. During a training session,each item presented to the front facing camera 122 of glasses 120 islabeled with its item code. The model configures itself during thetraining session to predict item codes when presented images without thecorresponding labels. Furthermore, when options for candidate items arepresented to customers during shopping sessions, the actual item codesselected by the customers are noted with the corresponding features ofthe item images and used in subsequent training sessions of the model tocontinuously improve item recognition accuracy of the model. In caseswhere the item barcode is visible in the images, the item barcode may beprovided with the item image for improving the accuracy of the model.

In an embodiment, the actual item images captured by camera 122 foritems are provided to item recognizer 114 for item recognition ratherthan select features. In this embodiment, the actual item images may bemodified for resolution to reduce the size of the item images beforebeing provided to item recognizer 114.

In an embodiment, item recognizer 114 may be processed on glasses 120and/or mobile device 130.

In an embodiment, glasses 120 include a high-resolution camera 122 of 12megapixels or more and glasses 120 include a depth sensor 125 associatedwith camera 122. Features of the item images captured by camera 122 and3D mapping of the item image captured by depth sensor 125 are processedon glasses 120 and/or mobile device 130 to derive a set of features(image and dimensions), which are provided to item recognizer 114 foritem recognition.

In an embodiment, an AR generated mapping of the store in connectionwith a planogram are used to provide location information for where eachitem image is captured within the store and the current location as wellas the planogram are provided as input with the item images to the itemrecognizer 114 for item recognition.

It is noted that although the wearable processing device 120 wasdiscussed as glasses 120, the wearable processing device 120 may begoggles 120 or an AR headset 120.

The above-referenced embodiments and other embodiments are now discussedwithin FIGS. 2-3 .

FIG. 2 is a diagram of a method 200 for managing a transaction sessionvia a wearable processing device, according to an example embodiment.The software module(s) that implements the method 200 is referred to asa “transaction session assistant.” The transaction session assistant isimplemented as executable instructions programmed and residing withinmemory and/or a non-transitory computer-readable (processor-readable)storage medium and executed by one or more processors of one or moredevices. The processor(s) of the device that executes the transactionsession assistant are specifically configured and programmed to processthe transaction session assistant. The transaction session assistant mayhave access to one or more network connections during its processing.The network connections can be wired, wireless, or a combination ofwired and wireless.

In an embodiment, the device that executes the transaction sessionassistant is wearable processing device 110. In an embodiment, thedevices that execute the transaction session assistant are wearableprocessing device 120 and mobile device 130.

In an embodiment, the transaction session assistant is all or somecombination of 127, 128, 129, 135, 136, and/or 114 (implemented ondevice 120 or device 130).

At 210, the transaction session assistant requests a connection to acloud-based store server 110 during a shopping trip of a customer to astore.

In an embodiment, at 211, the transaction session assistant makes theconnection based on a current location of a wearable processing device120 or a current location of a mobile device 130 paired to the wearableprocessing device 120 when the current location corresponds to a knownstore location for the store.

At 220, the transaction session assistant establishes a wirelesstransaction session to the cloud-based server 110 based on the requestof 210.

In an embodiment, at 221, the transaction session assistant establishesthe transaction session as a direct session between the wearableprocessing device 120 and the cloud-based server 110 (using wirelessconnection 140).

In an embodiment, at 222, the transaction session assistant establishesthe transaction session as an indirect session between the wearableprocessing device 120 and the cloud-based server 110 that utilizes asecond connection between the wearable processing device 110 and themobile device 130 (using wireless connection 160 with the mobile device130 using wireless connection 150 to the cloud-based store server 110).

At 230, the transaction session assistant captures item images placedwithin a field-of-view of a front-facing camera 122 of the wearableprocessing device 120 during the transaction session.

At 240, the transaction session assistant obtains item identifiers anditem information for the items associated with the item images based onat least the item images.

In an embodiment, at 241, the transaction session assistant extractfeatures for the item images, provides the features to the cloud-basedstore server 110 and receives the item identifiers back from thecloud-based store server 110.

In an embodiment of 241 and at 242, the transaction session assistantobtains a 3D mapping from a depth sensor 125 associated with thefront-facing camera 122 for each item image, provides the 3D mappingwith the item features to the cloud-based store server 110, and receivesthe item identifiers back from the cloud-based store server 110.

At 250, the transaction session assistant translates gestures of thecustomer during the transaction session into customer selections,customer options, and customer-initiated commands associated with avirtual shopping cart maintained by the cloud-based store server 110during the transaction session. The virtual shopping cart comprises theitem identifiers and item information for each of the items. Thegestures can be based on hand gestures made in front of the front-facingcamera 122, eye gestures made to a rear-facing camera 122, or headtilts/nods/swings detected by an accelerometer 123 of the wearableprocessing device 120.

In an embodiment, at 251, the transaction session assistant maps thegestures to interface selections, interface options, and interfacecommands of a transaction user interface associated with the cloud-basedstore server 110 that the cloud-based store server 110 processes toupdate and to maintain the virtual shopping cart during the transactionsession.

In an embodiment, at 252, the transaction session assistant furthertranslates audio spoken by the customer (captured by a microphone sensor125 of the wearable processing device 120) into text selections, textoptions, and text commands associated with the virtual shopping cart andprocessed by the cloud-based store server 110.

In an embodiment, at 260, the transaction session assistant is processedonly on the wearable processing device 120 or the transaction sessionassistant is processed on a combination of the wearable processingdevice 120 and a mobile device 130 in possession of or in proximity tothe customer during the transaction session.

In an embodiment, at 270, the transaction session assistant identifiesan image of a code captured within the field-of-view of the front-facingcamera 122. In an embodiment, the code is a QR code displayed on adisplay of a payment terminal (SST). In response to the captured code,payment processing for the items of the virtual shopping cart isinitiated with the virtual shopping cart.

In an embodiment of 270 and at 271, the transaction session assistantidentifies an authorization gesture or an audible command indicating thecustomer authorizes applying a registered payment method to complete thepayment processing and the transaction session assistant provides anauthorization to the cloud-based store server 110 to perform the paymentprocessing with the registered payment method and to end the transactionsession completing the shopping trip of the customer with the store.

In an embodiment of 270 and at 272, the transaction session assistantidentifies an authorization gesture or an audible command indicating thecustomer is ready for payment processing. The transaction sessionassistant captures an image of a payment card placed in thefield-of-view of the front-facing camera 122 and transaction the imageinto payment details. The transaction session assistant provides thepayment details to the cloud-based store server 110 to perform thepayment processing using the payment details. In another case, the imageof the payment card is provided by the transaction session assistant tothe cloud-based store server 110 and the cloud-based store servertranslates the image into the payment details and performs the paymentprocessing with the payment details.

FIG. 3 is a diagram of another method 300 for operating, conducting, andmanaging a transaction session with a wearable processing device,according to an example embodiment. The software module(s) thatimplements the method 300 is referred to as a “glasses shoppingmanager.” The glasses shopping manager is implemented as executableinstructions programmed and residing within memory and/or anon-transitory computer-readable (processor-readable) storage medium andexecuted by one or more processors of a device. The processors thatexecute the glasses shopping manager are specifically configured andprogrammed for processing the glasses shopping manager. The glassesshopping manager may have access to one or more network connectionsduring its processing. The network connections can be wired, wireless,or a combination of wired and wireless.

In an embodiment, the device that executes the glasses shopping manageris device 110. In an embodiment, the device that executes the glassesshopping manager is a combination of devices comprising 110, 120, andoptionally 130. In an embodiment device 120 is glasses, goggles, or anAR headset.

In an embodiment, the glasses shopping manager is some combination orall of 113, 114, 127, 128, 129, 135, 136, and/or method 200.

The glasses shopping manager presents another and, in some ways, anenhanced processing perspective from that which was shown above forsystem 100 and/or method 200.

At 310, the glasses shopping manager establishes a shopping session witha wearable processing device 120 (hereinafter AR-enabled glasses 120).The AR-enabled glasses 120 are worn by the customer in a store during ashopping trip. Lenses or a display of the AR-enabled glasses 120 permitthe customer to view their physical surroundings in the store along withAR information rendered as an overlay of the physical surroundings.

At 320, the glasses shopping manager maintains a virtual shopping cartfor the shopping session.

At 330, the glasses shopping manager receives item features for itemimages captured by a front-facing camera 122 of the AR-enabled glasses120 during the shopping session.

In an embodiment, at 341, the glasses shopping manager receives 3Dmeasurements for each item with the corresponding item features; the 3Dmeasurements captured by a depth sensor 125 associated with thefront-facing camera 122 of the AR-enabled glasses 120.

At 340, the glasses shopping manager resolves item identifiers for theitems based on at least the item features during the shopping session.

In an embodiment of 331 and 340, at 341, the glasses shopping managerscores the 3D measurements and the item features, and the glassesshopping manager matches scores produced against candidate scores forcandidate item identifiers to determine the item identifiers.

In an embodiment, at 342, the glasses shopping manager provides the itemfeatures to a trained machine-learning model (such as item recognizer114) and receives the item identifiers as output from themachine-learning model.

In an embodiment, at 343, the glasses shopping manager identifies acandidate list of item identifiers for at least one of the item imagesbased on the corresponding item features. The glasses shopping managerprovides the candidate item images and candidate item information forthe candidate list to the AR-enabled glasses 120 and receives agesture-selected option made by the customer to resolve particular itemidentifiers for each of the corresponding item images, which were unableto be initially determined by the glasses shopping manager.

At 350, the glasses shopping manager adds the item identifiers andcorresponding item details/information to the virtual shopping cartduring the shopping session.

In an embodiment, at 351, the glasses shopping manager provides asummary, a running price total, and a running item quantity total forthe virtual shopping cart to the AR-enabled glasses 120 for presentationin an AR-enabled display or AR-enabled lenses of the AR-enabled glasses120 during the shopping session.

At 360, the glasses shopping manager modifies the item identifiers or aquantity total for a given item identifier within the virtual shoppingcart based on gesture-based or audio-based input received from thecustomer through the AR-enabled glasses 120 during the shopping session.

At 370, the glasses shopping manager processes a payment for the itemsof the virtual shopping cart to conclude the shopping session based on apayment option communicated by the customer through the AR-enabledglasses 120.

It should be appreciated that where software is described in aparticular form (such as a component or module) this is merely to aidunderstanding and is not intended to limit how software that implementsthose functions may be architected or structured. For example, modulesare illustrated as separate modules, but may be implemented ashomogenous code, as individual components, some, but not all of thesemodules may be combined, or the functions may be implemented in softwarestructured in any other convenient manner.

Furthermore, although the software modules are illustrated as executingon one piece of hardware, the software may be distributed over multipleprocessors or in any other convenient manner.

The above description is illustrative, and not restrictive. Many otherembodiments will be apparent to those of skill in the art upon reviewingthe above description. The scope of embodiments should therefore bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

In the foregoing description of the embodiments, various features aregrouped together in a single embodiment for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting that the claimed embodiments have more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus, the following claims are herebyincorporated into the Description of the Embodiments, with each claimstanding on its own as a separate exemplary embodiment.

1. A method, comprising: requesting a connection to a cloud-based storeserver during a shopping trip of a customer to a store; establishing awireless transaction session to the cloud-based store server based onthe requesting; capturing item images placed within a field of view of afront-facing camera of a wearable processing device worn by the customerduring the transaction session; obtaining item identifiers and iteminformation for items associated with the item images based at least onthe item images; and translating gestures of the customer during thetransaction session into customer selections, customer options, andcustomer-initiated commands associated with a virtual shopping cartmaintained by the cloud-based store server during the transactionsession, wherein the virtual shopping cart comprises the itemidentifiers and item information.
 2. The method of claim 1 furthercomprising, processing the method on one or more of the wearableprocessing device worn by the customer and a mobile device in possessionof the customer or in proximity to the customer.
 3. The method of claim1 further comprising, identifying an image of a code captured within thefield-of-view of the front-facing camera, wherein the code displayed ona payment terminal of the store, and initiating payment processing forthe items of the virtual shopping cart.
 4. The method of claim 3 furthercomprising, identifying an authorization gesture or an audible commandindicating the customer authorizes applying a registered payment methodto complete payment processing and providing an authorization to thecloud-based store server to perform the payment processing with theregistered payment method to end the transaction session and completethe shopping trip of the customer to the store.
 5. The method of claim 3further comprising, identifying an authorization gesture or an audiblecommand indicating the customer is ready for payment processing,capturing an image of a payment card placed in the field-of-view of thefront-facing camera, translating the image into payment details, andproviding the payment details to the cloud-based store server to performpayment processing using the payment details to end the transactionsession and complete the shopping trip of the customer to the store. 6.The method of claim 1, wherein requesting further includes making theconnection based on a current location of the wearable processing deviceor a mobile device paired to the wireless processing device when thecurrent location corresponds to a store location for the store.
 7. Themethod of claim 1, wherein establishing further includes establishingthe wireless transaction session as a direct session between thewearable processing device and the cloud-based server.
 8. The method ofclaim 1, wherein establishing further includes establishing the wirelesstransaction session as an indirect session between the wearableprocessing device and the cloud-based server that utilizes a secondconnection between the wearable processing device and a mobile device.9. The method of claim 1, wherein obtaining further includes extractingfeatures of the item images, providing the features to the cloud-basedstore server, and receiving the item identifiers back from thecloud-based store server.
 10. The method of claim 9, wherein extractingfurther includes obtaining a three-dimensional (3D) mapping from a depthsensor associated with the front-facing camera for each of the itemimages, providing the 3D mappings with the features to the cloud-basedstore server, and receiving the item identifiers back from thecloud-based store server.
 11. The method of claim 1, wherein translatingfurther includes mapping the gestures to interface selections, interfaceoptions, and interface commands of a transaction user interfaceassociated with the cloud-based store server that the cloud-based storeserver processes to update the virtual shopping cart during thetransaction session.
 12. The method of claim 1, wherein translatingfurther includes translating audio spoken by the customer into textselections, text options, and text commands associated with the virtualshopping cart.
 13. A method, comprising: establishing a shopping sessionwith a wearable processing device worn by a customer in a store during ashopping trip; maintaining a virtual shopping cart for the shoppingsession; receiving item features for item images of items captured by afront-facing camera of the wearable processing device during theshopping session; resolving item identifiers for the items based atleast on the item features during the shopping session; adding the itemidentifiers to the virtual shopping cart during the shopping session;modifying the item identifiers or a quantity total for a given itemidentifier within the virtual shopping cart based on gesture-based oraudio-based input received from the customer through the wearableprocessing device during the shopping session; and processing a paymentto pay for the items of the virtual shopping cart and to conclude theshopping session based on a payment option communicated by the customerthrough the wearable processing device.
 14. The method of claim 13,wherein receiving further includes receiving three-dimensionalmeasurements for each item with the corresponding item features, whereinthe three-dimensional measurements captured by a depth sensor associatedwith the front-facing camera or the wearable processing device.
 15. Themethod of claim 14, wherein resolving further includes scoring thethree-dimensional measurements and the item features and matching scoresproduced against candidate scores for candidate item identifiers todetermine the item identifiers.
 16. The method of claim 13, whereinresolving further includes providing the item features to a trainedmachine-learning module and receiving the item identifiers as outputfrom the trained machine-learning module.
 17. The method of claim 13,wherein resolving further includes identifying a candidate list of itemidentifiers for at least one item image received based on thecorresponding item features, providing candidate item images andcandidate item information for the candidate list to the wearableprocessing device, and receive a gesture-select option made by thecustomer to resolve a particular item identifier for each of the atleast one item images.
 18. The method of claim 13, wherein modifyingfurther includes providing a summary, a running price total, and arunning item quantity total for the virtual shopping cart to thewearable processing device for presentation to the customer on anAugmented Reality (AR)-enabled display or lenses of the wearableprocessing device during the shopping session.
 19. A system, comprising:at least one server comprising a processor and a non-transitorycomputer-readable storage medium; and a wearable processing devicecomprising an Augmented Reality (AR)-enabled display or AR-enabledlenses, a front-facing camera, a rear-facing camera, a wirelesstransceiver, a microphone, a depth sensor, and an accelerometer; thesystem configured to establish a wireless transaction session betweenthe server and the wearable processing device during a shopping trip ofa customer to a store using the wireless transceiver of the wearableprocessing device; the wearable processing device, during thetransaction session, is configured to: capture item images of itemsplaced within the field-of-view of the front-facing camera; extractfeatures for the item images; obtain three-dimensional measurements foreach item image using the depth sensor; provide the features andthree-dimensional measurements to the server; confirm item identifiersfor the items added to a virtual shopping cart by the server; displaycart and item information on the AR-enabled display or the AR-enabledlenses; display feedback information on the AR-enabled display or theAR-enabled lenses received from the server; translate gestures made bythe customer and detected by the front-facing camera, the rear-facingcamera, and accelerometer into transaction interface commands,transaction interface selections, and transaction interface optionsrecognized by the server; and translate audio spoken by the customercaptured by the microphone into the transaction interface commands, thetransaction interface selections, and the transaction interface options.the server, during the session, configured to: identify the itemidentifiers from the features and the three-dimensional measurementsassociated with the item images; obtain candidate item images when agiven item image corresponds to multiple candidate item identifiers;confirm each item identifier added to, modified, or removed from thevirtual shopping cart through the transaction interface commands, thetransaction interface selections, and the transaction interface optionsreceived from the wearable processing device; maintain the virtualshopping cart; provide the feedback information to the wearableprocessing device for the virtual shopping cart and for resultsassociated with processing the transaction interface commands, thetransaction interface selections, and the transaction interface options;and processing a payment for the virtual shopping cart based on selectones of the transaction interface commands to end the transactionsession.
 20. The system of claim 19, wherein the wearable processingdevice is glasses or a headset.